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A  FORMAL  ANALYSIS  OF  THE  EVOLUTION  OF  COOPERATION 


I.  INTRODUCTION 


In  this  paper, 'we  examine  a  formal  model  of  the  way  a  growing 
population  selects  behavior.  The  members  of  the  population  engage  in 
randomly-selected  binary  interactions,  with  payoffs  representing  a 
Prisoners'  Dilemma.  We  assume  that  players  can  recognize  each  other  if 
they  have  met  before,  and  adopt  one  of  two  dynamic  strategies;  they 
either  play  the  non-cooperative  strategy  (G)  at  every  move,  which  we 
denote  D,  or  they  play  a  tit-for-tat  strategy,  in  which  they  begin  by 
playing  the  cooperative  move  (C)  and  continue  by  playing  C  if  the 
opponent's  last  move  was  C  and  G  if  the  opponent's  last  move  was  G.  This 
latter  strategy  is  denoted  T.  The  interactions  are  repeated,  and  two 
players  meet  again  with  probability  6  e  (0,1).  Formally,  it  is  as  if 
each  player  met  a  random  member  of  the  population  and  played  a  J 

discounted  supergame  with  discount  factor  6  forever  afterwards.  An 
alternative  interpretation  is  that  players  repeatedly  meet  random 
opponents,  but  respond  to  each  opponent  they  have  met  before  on  the 
basis  of  what  happened  during  their  previous  encounter. 

We  assume  that  the  population  and  the  mix  of  behaviors  changes  over 
time  in  a  simple  fashion.  If  one  of  the  two  strategies  D  or  T  has  a 
higher  expected  present  discounted  value  (or  "fitness")  than  the  other, 
the  proportion  of  players  using  that  strategy  will  tend  to  increase  and 
the  proportion  using  the  less  fit  strategy  will  tend  to  diminish.  The 
same  conflict  mediates  average  as  well  as  relative  fitness.  The 
relative  proportions  of  D  and  T  in  the  population  determine  the  average 
expected  payoff,  ♦  ,  and  the  rate  of  growth  of  the  total  population  is 
assumed  to  be  a  monotone  increasing  function  of  ♦.  Moreover,  there 
exists  a  critical  value  ♦*  such  that  a  population  with  ♦  >  ♦*  will 
expand,  while  a  population  with  average  fitness  #  <  ♦*  will  contract. 

These  qualitative  dynamics  will  allow  us  to  say  something  about  the 
stability  of  various  regimes  of  behavior  in  terms  of  population  size  and 
behavior . 


-  The  organization  of  the  paper  is  as  follows.  Section  II  contains 
the  model  and  definitions,  and  examines  the  evolution  of  cooperative 
behavior  for  the  special  case  of  a  static  population.  Section  III 
combines  the  dynamics  of  behavior  with  those  of  population  growth. 
Finally,  Section  IV  relates  the  present  results  to  results  of 
repeated  play  where  players  are  rational. 
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II.  THE  MODEL  AND  DEFINITIONS 

Following  Axelrod  (1984),  we  assume  that  single  interactions  are 
mediated  by  a  generalized  Prisoners'  Dilemma: 

+ - + - + 

C  | (b,b) | (d,a) | 

+ - + - + 

G  | (a,d) | (c,c) | 

+ - + - + 

where  a  >  b  >  c  >  d,  and  2c  <  a+d  <  2b,  so  the  players  cannot  escape 
their  dilemma  by  randomizing.  The  size  of  the  population  is  denoted  n. 

In  this  model,  players  are  paired  randomly,  and  the  result  of  these 
matches  determines  the  fitness  of  the  strategies  they  select.  If  each 
player  plays  only  once,  these  strategies  are  simply  "moves"  C  or  G.  As  G 
is  a  dominant  strategy,  a  population  of  players  who  met  only  once  would 
eventually  all  play  G. 

To  make  the  dynamics  interesting,  we  shall  assume  that  players  may 
meet  repeatedly  during  their  lifetimes.  This  assumption  changes  the 
strategies  used  by  the  players  and  the  way  they  evaluate  outcomes.  At 
the  strategic  level,  players  are  allowed  to  make  moves  that  depend  on 
previous  history.  In  this  note,  we  shall  limit  ourselves  to  two  very 
simple  forms  of  historical  dependence.  However,  we  shall  demonstrate  in 
Section  V  that  the  implications  for  the  viability  of  cooperation  are  not 
grossly  inaccurate.  At  the  payoff  level,  the  possibility  of  repeated 
encounters  means  that  the  players  consider  the  entire  stream  of  payoffs 
resulting  from  their  behavior,  which  determines  their  ultimate  fitness. 

We  shall  suppose  that  players  discount  the  future  acording  to  a  discount 
factor  6  t  (0,1).  There  are  various  interpretations  of  6.  In  one  view, 

6  represents  the  (constant)  probability  that  players  meeting  today  will 
meet  again  in  the  future.  Alternatively,  if  we  imagine  that  fitness  of 
an  organism  during  its  lifetime  is  determined  by  cumulative  payoff,  6 


-  4  - 

may  be  interpreted  as  the  probability  that  a  player  who  has  not 
reproduced  will  reproduce  on  a  given  date.  In  our  simplified 
evolutionary  view  of  the  world,  the  moment  of  reproduction  is  the  moment 
of  death,  since  an  organism's  importance  to  the  species  ends  when  it 
passes  on  its  behavior. 

The  two  strategies  we  shall  consider  here  are:  a  noncooperative 
strategy  D  calling  for  the  player  to  play  G  every  time  it  is  called  upon 
to  play,  no  matter  who  its  opponent  may  be;  and  a  Tit-for-Tat  strategy  T 
calling  for  an  organism  to  begin  by  playing  C,  and  to  continue  by 
playing  C  whenever  the  other  player  played  C  on  the  previous  move  and  G 
otherwise . 1 

For  concreteness,  we  shall  describe  a  population  in  terms  of  the  number, 
g,  of  its  members  using  strategy  D. 

If  two  D  players  meet,  the  present  discounted  value  each  receives 
is  h(D,D)  =  c/(l-6).  If  two  T  players  meet,  the  present  discounted 
value  each  receives  is  h(T,T)  =  b/(l~6).  Finally,  if  a  D  player  meets  a 
T  player,  their  respective  present  discounted  values  are: 

h(D,T)  =  [ (l-6)a  +  6c] / (1-6) 

(1) 

h(T,D)  =  ( (l-6)d  +  6c]/(l-5) 

since  the  Tit-for-Tat  player  will  discover  that  his  opponent  is  a  D 
player  after  one  move.  On  the  assumption  that  opponents  are  randomly 
selected,  we  may  define  the  fitness  of  a  strategy  to  be  its  expected 
present  discounted  value,  given  n  and  g: 

F(D;n,g)  =  prob(D)xh(D,D)  +  prob(T)xh(D,T) 

=  ((g-l)c  +  (n-g) ( 1 -6)a  +  (n-g)6c]/[ (n-l)(l-6)] 

F(T;n,g)  =  prob(D)xh(T,D)  +  prob(T)xh(T,T) 

=  |g(l-6)d  +  g6c  +  (n-g-l)b)/[ (n-1) (1-6) ] 


(2) 


The  average  fitness  of  the  population  is: 

(3)  *(n,g)  i  [gF(D;n,g)  +  (n-g)F(T;n,g) ]/n 

Definition:  a  population  (n,g)  is  in  evolutionary  equilibrium  with 
respect  to  a  critical  fitness  #*  iff  one  of  the  following  conditions 
holds : 

1  n  =  0 

2  $(n,g)  =  #*,  g  =  n,  and  F(D;n,g)  Z  F(T;n,g) 

3  #(n,g)  =  #*,  g  =  0,  and  F(T;n,g)  >  F(D;n,g) 

4  #(n,g)  =  i*,  g  e  (0,n),  and  F(T;n,g)  =  F(D,n,g) 

These  are  all  stationary  equilibria.  The  first  represents 
extinction  of  the  species.  The  second  represents  a  stable  population 
that  is  completely  non-cooperative.  The  third  represents  a  stable 
population  which  is  completely  cooperative,  and  the  fourth  represents  a 
stable  population  in  which  a  mixture  of  cooperative  and  noncooperative 
behaviors  survives.  In  addition  to  these  outcomes,  there  are  equilibria 
involving  continued  population  growth.  Along  these  "dynamic 
equilibrium"  paths,  cooperation  may  either  increase  or  decrease. 

For  the  balance  of  this  section,  we  shall  assume  that  the  total 
population  is  fixed,  so  we  will  not  need  to  use  f(n,g). 

The  dynamics  of  behavior  are  determined  by  relative  fitness.  That 
is,  we  assume  that  g  increases  over  time  iff  F(D;n,g)  >  F(T;n,g).  The 
set  of  populations  for  which  g  is  constant  is  determined  by  the 
condition  F(D;n,g)  =  F(T;n,g)  in  case  g  e  (0,n).  Alternatively,  g  is 
constant  if  g  =  n  and  F(D;n,n-l)  >  F(T;n,n-l)  or  if  g  =  0  and 
F(D;n,l)  <  F(G;n,l).  Strictly  speaking,  this  gives  conditions  for  a  single 
mutant  to  die  out,  and  the  definitions  do  not  require  that  such  a  mutant 
never  appear.  In  our  model  there  is  no  possibility  of  spontaneous 
mutation,  so  these  conditions  are  sufficient.  In  any  event,  g  =  n  is 
always  constant,  and  the  analysis  of  F(D;n,g)  =  F(T;n,g)  which  we  shall 
present  gives  a  more  precise  condition  for  g  =  0  to  be  constant. 


There  are  manv  ways  to  present  the  condition  of  equal  fitness.  For 
the  present,  we  shall  give  the  condition  in  terms  of  6  and  g,  since  n  is 
fixed  in  tl is  section.  Let  us  define  6*(g)  by: 

(4)  6*(g)  =  [ (n-g) (a-b)  +  g(c-d)  +  (b-c) ] / [ (n-g) (a-c)  +  g(c-d)] 

Clearly,  6*(0)  =  [n(a-b)+(b-c) ] / (n(a-c) ]  <  1  since  b  >  c.  Moreover,  if 
we  write  6*(g)  =  f(g)/h(g),  we  note  that  0  <  f(g)  £  h(g)  for  g  <  n  and 
also  that  f'(g)  >  h'(g).  Therefore,  6*(g)  is  a  monotone  increasing 
function  of  g.  Indeed,  6*(n-l)  =  1,  so  the  relative  fitness  of  the  two 
strategies  is  as  shown  in  Figure  1. 


Thus,  populations  starting  above  the  indicated  line  will  tend  to 
become  completely  cooperative,  while  populations  starting  below  the  line 
will  become  completely  noncooperative.  This  diagram  also  tells  us  that 
full  cooperation  can  be  sustained  if  fi  >  6*(0) .  It  can  be  sustained  in 
the  face  of  spontaneous  mutation  of  a  given  degree  if  6  exceeds  6*(0)  by 
a  sufficient  amount.  As  we  shall  show  in  Section  V,  this  condition  is 
generally  stricter  than  the  condition  that  cooperation  can  be  sustained 
as  an  equilibrium  with  arbitrary  strategies:  6  £  (a-b)/(a-c).  However, 


Fig.  1  --  Behavior  Dynamics  with  Fixed  Population 


as  n 


■*  ~,  these  conditions  become  identical.  One  might  therefore  say 
that  cooperation  is  sustainable  as  an  equilibrium  with  arbitrary 
strategies  iff  it  is  evolutionarily  stable  (in  this  sense)  in  an 
infinite  population. 

We  can  also  examine  the  influence  of  population  size  on  the 
stability  of  cooperation.  Equation  (4)  can  be  written: 

(5)  g  =  g*(n)  =  [on  -  (b-c)]/<fi,  where 

o  =  b  -  (l-6)a  -  6c,  and 

=  b  +  c-  a-  d  +  6(a  +  d  -  2c) 

We  always  have  a  <  <f>.  Moreover,  o  >  0  iff  6  >  6  ,  and  >  0 
iff  6  >  6 , ,  where: 

(6)  6^  5  [a+d-b-c] / [a+d-2c]  <  [a-b]/[a-c)  5 

with  the  inequality  following  from  c  >  d.  There  are  thus  three 
possibilities : 


6  <  6^  --  in  which  case  g*  has  a  positive  g-intercept  and 
slope  >  1; 

6^  <  6  <  --  in  which  case  g*  has  a  negative  g-intercept  and 

slope  <  '1,  and 

6o  <  6  --  in  which  case  g*  has  a  negative  g-intercept  and 
positive  slope  <  1. 


> 


will 


To  interpret  these  in  terms  of  relative  fitness,  we  observe  that  g 
increase  iff  \(ig  >  an  -  (b-c).  Therefore: 
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if  6  <  6^,  g  increases  iff  g  <  g*(n);  and 
if  6  >  6^,  g  increases  iff  g  >  g*(n) 


Since  the  .easible  set  of  populations  is  {(n,g):0  <  g  <  n},  this 
means  that  g  is  increasing  everywhere  if  6  <  6q.  For  6  >  6o>  the 
dynamics  of  g  with  fixed  n  are  as  shown  in  Figure  2. 

Therefore,  with  a  fixed  population  size  increasing  myopia  and  small 
numbers  both  act  to  make  cooperation  less  viable. 


I 


Fig.  2  --  Behavioral  Dynamics  with  Fixed  n  and  6  >  6 


III.  POPULATION  GROWTH  AND  BEHAVIORAL  DYNAMICS 


In  this  section,  we  shall  combine  the  previous  analysis  of  behavior 
selection  with  an  analysis  of  population  growth.  In  doing  so,  we  assume 
that  the  rate  of  growth  of  population  is  a  strictly  monotone  increasing 
function  of  average  fitness  #(n,g)  defined  in  (3)  above.  This  is 
certainly  not  the  most  general  assumption;  indeed,  one  can  think  of  many 
forms  of  ritual  combat  for  which  the  effects  on  individual  fitness  and 
species  fitness  are  opposed.  However,  this  assumption  serves  to 
simplify  the  analysis. 

Since  we  are  using  qualitative  dynamics,  the  condition  for  nondecreasing 
population  size  is  that  average  fitness  exceed  a  critical  level  <  b: 

2 

(7)  #(n,g)  -  i*  =  *g  +  ug  +  t ($*)  £  0,  where 

w  =  (l-6)n(a+d)  -  (2n-l)b  +  (2n6-l)c 
T(**)  =  n(n-l) (b  -  #*) 

Setting  this  expression  equal  to  0,  we  obtain  two  possible  values 
of  g  at  which  n  is  constant: 

+  2  1/2 
h  (n)  =  [ -u  +  (w  -  4*i(f*))  1/2*,  and 

(8) 

h" (n)  =  [-w  -  (w2  -  4*t ($*))1/2]/2* 

Let  g+(n)  =  max{h+(n),h  (n)}  and  g  (n)  =  min{h+(n),h  (n)}. 

•  n  will  be  constant  iff  n  =  0,  g  =  g+(n),  or  g  =  g  (n). 

•  If  6  >  6^,  n  will  be  increasing  for  g  <  g  (n)  and  for 

g  >  g+(n),  but  will  be  decreasing  for  g  (n)  <  g  <  g+(n). 
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•  If  6  <  6^,  n  will  be  decreasing  for  g  <  g  (n) 
g  >  g+(n) ,  but  will  be  increasing  for  g  (n)  < 

Proposition:  i)  {  implies  h+(n)  £  n; 

ii)  6  £  6  implies  h+(n)  <  0 

P 

iii)  h  (n)  i  0  if 

1  6  £  6  ;  or 

P 

2  6  £  6  and  n  £  1 ; 

P 

iv)  h  (n)  <  n  if  n  £  1. 

proof:  i)  6  £  6  implies  ip  £  0.  The  condition  h+(n) 

P 

equivalent  to: 

[w2  -  4^t(**)]^2  £  u  +  2^r (■♦*),  or 
n  £  [♦*  -  c]/(**  +  (l-26)o] 

which  is  always  true  for  6  c  [0,1]. 

ii)  6  S  6  implies  <  0.  The  condition  h+(n)  £  0  is 

P 

[w2  -  4*T(f*)]1/2  &  w 

which  is  certainly  true  if  w  <  0.  On  the  other  hand, 
equation  can  be  written: 

0  £  4*t(**) 

which  is  always  true. 

iii)  6  £  6  implies  ^  ^  0,  so  the  condition  h  (n)  >  0 

P 

-w  2  [w2  -  4*t(**)]1/2 


and  for 
g  <  g  (n). 


>  n  is  then 


then  equivalent  to: 


if  •<,  £  0,  the  above 


becomes : 


which  is  true  if  w  £  0.  However,  w  £  0  iff  n  £  N^,  where 

NQ  =  [b-c]/[2b  -  (a+d)  +  6(a+d-2c)] 

Moreover,  i  1  iff  £  0.  Hence  6  i  6  implies 
$  >  0,  which  implies  >  1.  Therefore,  n  >  1  implies  n  i  N^, 

hence  «  £  0,  and  therefore  h  (n)  £  0. 

On  the  other  hand,  if  5  <  5  ,  ^»  <  0,  and  the  condition  h  (n)  £  0 

P 

is: 

-w  <  [w2  -  4*t(**)]1/2 

which  is  certainly  true  if  u  £  0.  Indeed,  if  u  <  0,  we  can  square  both 

sides  of  the  above  condition  to  give  4<f>t(f*)  £  0,  which  follows  from 

6  <  6  . 

P 

iv)  First,  suppose  that  M  0;  the  condition  h  (n)  £  n  is: 

-  [«2  -  4*t(**)]1/2  £  2^n  +  w,  or 
2 

0  >  t(4*)  +  tfn  +  wn,  or 
0  <  (#*  -  c)  +  (1-6) (a+d) 

which  is  clearly  true. 

If  ♦  <  0,  h  (n)  £  n  only  if  a)  2<pn  +  u  £  0  and 
b)  -[w2  -  4^x (i*)]^^2  i  2tl>n  +  w.  a)  is  true  iff 

n  £  [b  -  c)/( (l-6)(a  +  d  -  2c)] 

which  is  <  1  iff  4>  <  0.  Therefore,  n  2  1  implies  a),  while  b)  is: 

0  £  (1-6) (a+d)  +  (♦*  -  c) 


which  is  true  by  assumption. 


QED 


There  are  various  types  of  candidate  equilibria  for  this  process. 
There  are  internal  equilibria  at  which  g  =  g  (n)  =  g*(n),  which  have 
constant  populations  with  a  mixture  of  behaviors.  There  are  also 
extinction  equilibria  in  which  n  approaches  0,  and  expansion  equilibria 
in  which  n  tends  to  infinity.  The  behavior  associated  with  each  type  of 
equilibrium  path  will  depend  on  the  height  of  the  critical  fitness  among 
other  things . 

Before  illustrating  these  possibilities,  we  shall  discuss  the 

possibility  of  cyclical  behavior.  An  equilibrium  in  which  g  e  (0,n) 

represents  the  intersection  of  g*(n)  with  g  (n) . .  From  equation  (11),  we 

infer  that  g*  passes  through  the  feasible  range  only  if  6  >  6  >  6  . 

o  p 

The  derivative  of  g  w.r.t.  n  is  nonnegative  iff: 

(9)  4*x  (**)  i  w  [2  +  (w2+4*t(**))1/2],  where 

n  n 

=  (1*6) (a+d)  -  2b  +  26c,  and 
xn(**)  =  (2n-l)(b-**) 

which  follows  from  the  facts  that  a+d  <  2b  and  ♦*  £  b.  In  fact,  this 
formula  can  also  be  used  to  show  that  g  (n)  is  strictly  concave.  The 
phase  diagrams  corresponding  to  possible  intersections  of  g*  and  g  are 
shown  in  Figure  3.  Both  g*  and  g  are  increasing.  If  g*  crosses  g 
from  above,  the  equilibrium  at  e  is  only  approached  along  the  separatrix 
S,  and  all  other  paths  ultimately  diverge  monotonical ly  from  e.  If  g* 
crosses  g  from  below,  e  is  completely  unstable,  and  all  paths  diverge 
in  an  ultimately  monotone  fashion. 

On  the  other  hand,  by  continuity  any  cyclical  behavior  must  occur 
around  an  equilibrium  of  g*(n)  and  g  (n) ,  so  there  can  be  no  cycles. 


r 
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Fig.  3  --  Qualitative  Dynamics  near  an  Interior  Equilibrium  of  g*  and  g_ 

We  shall  now  illustrate  the  possible  equilibrium  behavior  in  the 
context  of  a  specific  example. 


THE  EXAMPLE 

This  example  is  defined  by  the  following  data: 


a  =  4 ,  b  =  3 ,  c  =  1 ,  d  =  0 


Under  these  conditions,  the  critical  values  of  6  are 


6  =  (a+d-b-c)/ (a+d-2c)  =  0,  and  6  =  (a-b)/(a-c)  =  1/3 

P  o 


The  functions  defining  the  behavior  of  g  and  n  are: 


g*(n)  =  (36-l)n/26  -  1/6 


h+(n)  =  [n( 1+6 ) - 1 J /26  +  [(n(l+6)-l)2  -  26n(n-l) (3-**) J 1/2/26 
h" (n)  =  |n(l+6)-l)/26  -  [(n(l+6)-l)2  -  26n(n-l) (3-**) ) 1/2/26 


y-i- v  >  s v-v--;  -ix.. 


We  shall  examine  three  versions  of  this  model,  corresponding  to 
three  possible  values  for  ♦*. 

Case  1 :  **  =  3 

In  this  case,  full  cooperation  is  essential  for  survival.  This  is 
reflected  in  the  behavioral  equations: 

g+(n)  =  max{n  +  (n-l)/6,  0};  g  (n)  =  min{n  +  (n-l)/6,  0} 

The  phase  diagram  is  shown  in  Figure  4.  Populations  starting  above 
the  separatrix  labelled  "S"  are  doomed  to  extinction,  since  they  cross 
the  g*  locus  before  reaching  full  cooperation.  Once  they  reach  g*(n), 
cooperation  begins  to  erode,  and  the  species  must  decline.  Populations 
starting  below  S  will  survive  as  totally  cooperative  (g  =  0) 
populations.  Their  size  is  determined  by  the  point  at  which  they  hit 
the  n-axis. 

The  size  and  position  of  the  region  of  viable  populations  is 
determined  by  the  discount  factor  6  (as  well  as  by  the  unspecified 
relative  rates  of  adjustment  of  g  and  n) .  As  6  grows,  the  set  of  viable 
populations  expands,  since  g*  becomes  steeper,  approaching  n-(l/2).  If 
the  population  becomes  more  myopic,  g*  becomes  flatter  and  intersects 


Fig.  4  --  Population  Dynamics  for  Case  1:  ♦*  =  3,  6  >  1/3 
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the  n-axis  at  a  greater  total  population.  Indeed,  for  6  £  1/3,  the  only 
outcome  is  extinction. 

It  should  be  mentioned  that  there  is  a  region  near  the  origin  that 
lies  below  the  45®  line  but  above  g+(n).  This  region  only  includes 
populations  of  size  less  than  one.  However,  populations  within  this 
region  do  enjoy  positive  fitness,  and  to  be  strictly  accurate  we  should 
say  that  all  populations  which  do  not  reach  a  completely  cooperative 
equilibrium  reach  a  completely  noncooperative  equilibrium  with  n  =  1. 
However,  this  may  be  considered  extinction  in  our  model. 

Case  2:  ♦*  =  1 

This  case  represents  the  other  extreme:  even  completely 
noncooperative  behavior  is  viable.  In  this  case,  the  behavioral 
dynamics  are  determined  by: 

+ 

g  (n)  =  max{ (n-l)/6 ,  n} ;  g  (n)  =  min{ (n-l)/6 ,  n) 

The  phase  diagram  is  shown  in  Figure  5.  Populations  beginning  in 
region  A  will  become  extinct,  following  a  noncooperative  path. 
Populations  in  region  B  will  reach  a  noncooperative  equilibrium  where 
their  trajectory  touches  the  45*  line.  Populations  starting  in  region  C 
will  grow  forever,  with  a  noncooperative  population  whose  relative 
magnitude  is  bounded  below  by  (36-l)/26.  Populations  in  region  D  will 
grow  monotonically ,  but  the  number  of  noncooperative  players  will  first 
increase  until  the  trajectory  reaches  g*(n)  and  then  decrease 
monotonically  to  0.  Finally,  populations  in  region  E  will  grow  and 
become  completely  cooperative. 


Case  3:  **  =  2 

This  case  represents  an  intermediate  level  of  "selection,"  and 
displays  more  interesting  behavior.  The  g+(n)  locus  lies  entirely  above 
the  45®  line  for  nil,  and  the  g  (n)  locus  is  given  by: 


g 


-g 


A5#  (g') 


Fig.  5  --  Population  Dynamics  for  Case  2:  #*  =  1,  6  >  1/3 

g"(n)  =  [n(l+6)  -  1  -  [ (l+62)n2  -  2n  +  l)1/2]/26 

This  is  a  concave  monotone  increasing  curve  passing  through  (n,g)  = 

2  1/2 

(1,0),  and  asymptotic  to  [1+6  -(1+6)  ]26.  This  curve  intersects 

1/2 

g*(n)  as  long  as  6  >  2- (3)  '  ,  and  is  labelled  "e"  in  the  phase  diagram 
presented  in  Figure  6. 


Populations  lying  above  the  separatrix  S  will  become  extinct.  If 
they  lie  above  g*  and  g  ,  the  population  will  shrink  monotonical ly  while 
the  proportion  of  noncooperative  players  increases.  If  the  population 


Fig.  6  --  Population  Dynamics  for  Case  3:  ♦*  =  2,  6  >  1/3 


lies  above  S  but  below  g*,  population  will  shrink  monotonically ,  while 
the  degree  of  cooperation  first  increases  and  then  decreases.  Finally, 
populations  starting  above  S  but  below  g  will  have  a  monotonically 
increasing  proportion  of  noncooperative  players.  Total  population  will 
increase  initially,  but  then  decline  to  0. 

Populations  starting  on  the  separatrix  will  converge  monotonically 
to  the  "mixed-behavior"  equilibrium  at  "e" .  This  is  an  unstable  path  of 
the  sort  familiar  to  growth  economists.  Random  increases  in 
noncooperation  will  lead  to  extinction;  random  increases  in  cooperation 
will  lead  to  continued  cooperative  growth. 

Finally,  populations  starting  below  S  will  eventually  expand,  and 
the  degree  of  cooperation  will  eventually  become  absolute.  However,  if 
such  a  population  starts  above  g*  there  will  be  an  initial  period  of 
increasing  noncooperation.  On  the  other  hand,  a  population  starting 
above  S  but  below  g  will  experience  an  initial  decline  in  population 
followed  by  growth. 


IV.  RELATION  TO  NASH  EQUILIBRIUM  IN  THE  SUPERGAME 


The  concept  of  evolutionary  equilibrium  used  here  has  certain 
relations  to  equilibrium  concepts  used  in  the  game  theoretic  literature. 
These  relations  can  shed  light  on  the  validity  of  the  assumption  that 
the  only  two  possible  strategies  are  D  and  T.  In  this  discussion  we 
assume  a  fixed  population  size. 

A  more  general  definition  of  evolutionary  dynamics  would  begin  by 
specifying  a  set,  Z,  of  possible  strategies  to  be  used  in  a  binary 
symmetric  game  with  payoff  function  h.  Assuming  that  each  member  of  the 
population  uses  a  pure  strategy  o  z  Z,  a  population  strategy  s  e  S  is 
defined,  where  S  is  the  set  of  probability  measures  on  Z.  For  a  finite 
population,  the  feasible  population  strategies  will  form  a  finite  subset 
of  S. 


For  a  game  with  a  finite  population  n,  s(o)  will  be  a  rational 
number  of  the  form  s(o)  =  m(o)/n,  where  m(o)  is  the  number  of 
individuals  using  strategy  a.  In  an  infinite  game,  s(o)  is  the 
proportion  of  the  population  using  0.  Using  these  interpretations,  we 
can  define  the  fitness  of  a  pure  strategy  o  against  s  in  a  game  of  size 
n  as : 


F(o,s;n)  =  [ (m(o ) - 1 )h(o ,o )  +  Z  m(t )h(o , T ) ] / (n- 1 )  for  n  <  «; 

x^o 


F(o  ,s  ;«*)  =  Z  s(t  )h(o  ,x  ) 

T 

An  evolutionary  dynamical  system  is  an  equation  G:S  -*  S  such  that 
s(o),  s(t)  >  0,  F(o,s;n)  >  F(i,s;n)  implies  G(s)(o)s(t)  >  G(s)(x)s(o). 

An  evolutionary  equilibrium  is  a  rest  point  of  an  evolutionary  dynamical 
system.  This  is  a  very  broad  definition  of  an  evolutionary  equilibrium, 
that  includes  all  Nash  equilibria  of  the  game  if  the  population  is 
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infinite,  and  all  feasible  Nash  equilibria  if  not.  In  fact,  in  the 
finite  case  it  includes  some  outcomes  that  are  not  Nash  equilibria. 
Another  interesting  point  is  that  an  evolutionary  dynamical  system  may 
give  rise  to  cyclical  behavior  in  the  neighborhood  of  an  infeasible  Nash 
equilibrium. 

Tighter  definitions  of  evolutionary  equilibrium  can  be  obtained  by 
adding  stability  considerations.  For  example,  we  might  wish  to  add  the 
condition  that  the  proposed  equilibrium  behavior  should  be  approachable 
from  a  specified  set  of  initial  conditions,  or  that  it  should  be  stable 
under  a  specified  set  of  perturbations.  This  gives  the  evolutionary 
concept  some  of  the  flavor  of  perfect  equilibrium  concepts. 

For  our  purposes,  it  suffices  to  observe  that  evolution  from  a 
completely  mixed  starting  point  may  be  equivalent  to  sequential 
elimination  of  dominated  strategies.  If  a  given  pure  strategy  is 
dominated  by  another  pure  strategy,  the  number  of  individuals  using  the 
dominated  strategy  will  eventually  decline  in  a  completely  mixed 
population  following  ovolutionary  dynamics.  In  fact,  it  will  become 
extinct  before  the  dominating  strategy.  Eventually,  we  can  look  forward 
to  a  situation  in  which  all  strategies  dominated  by  pure  strategies  have 
been  eliminated  from  the  population.  At  this  point,  the  same  argument 
can  be  applied  repeatedly.  However,  there  is  nothing  in  the  qualitative 
dynamics  to  ensure  that  the  first  strategies  to  go  will  all  be  dominated 
strategies.  Therefore,  since  order  is  important  iri  determining  the 
results  of  sequential  elimination  of  dominated  strategies,  we  cannot  use 
the  idea  of  starting  from  a  completely  mixed  strategy  to  conclude  that 
only  sequentially  undominated  strategies  will  appear,  although  it  does 
ensure  that  any  evolutionary  equilibrium  established  in  this  way  is 
perfect . 

However,  it  remains  true  that  there  exist  evolutionary  dynamical 
systems  which  have  the  property  that  dominated  strategies  are  eliminated 
before  undominated  strategies  when  the  population  starts  from  a 
completely  mixed  position.  Using  such  a  system,  one  can  infer  that 
evolutionary  equilibrium  will  involve  sequentially  undominated 


strategies.  In  the  undiscounted  Prisoner's  Dilemma  in  which  player's 
strategies  are  allowed  to  depend  only  on  the  opponent's  previous  move, 
the  strategies  D  and  T  are  the  last  strategies  to  be  eliminated.* 

In  that  game,  T  is  the  only  surviving  strategy  at  the  last  step,  but 
with  discounting  T  does  not  dominate  D.  This  provides  indirect  evidence 
that  a  population  that: 

•  is  limited  to  remembering  the  opponent's  previous  move; 

•  begins  with  a  complete  mix  of  strategies;  and 

•  follows  dynamics  which  result  in  relatively  early  elimination 
of  dominated  strategies 


will  eventually  conform  to  the  assumptions  of  the  model  used  here. 


FOOTNOTES 


1 . In  the  version  of  the  game  where  players  cannot  recognize  each  other, 
a  Tit-for-Tat  player's  move  is  based  on  the  previous  move  of  whichever 
opponent  it  faced  previously.  Of  course,  in  this  model  there  are  no 
continuing  relationships,  and  the  players  face  random  opponent  selection 
at  each  stage. 

2.  See  Cave  (1984). 
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